Digital Watermarking Applications

ABSTRACT

Methods and systems include, e.g.,: (1) steganographically embedding location information in images, where the location information is obtained from remote sources like a cell phone network or remote GPS receiver; (2) steganographically embedding participant IDs in content to ensure proper billing and royalty tracking; (3) providing fair-use content management based upon digital watermark-tracked usage; (4) providing micro-payments based upon watermarked ID cards for retailers; and/or (5) providing watermarked logon cards, such as watermarked hotel room keys, to better provide internet logon access control. Other systems and methods are provided as well.

RELATED APPLICATION DATA

This application is a division of U.S. patent application Ser. No.10/622,079, which claims the benefit of U.S. Provisional PatentApplication No. 60/396,893, filed Jul. 16, 2002. These patent documentsare hereby incorporated herein by reference.

TECHNICAL FIELD

The invention relates to digital watermarking, digital fingerprinting,and the use of such technology for copy protection, digital assetmanagement, access control, authentication, content monitoring, and avariety of other applications.

BACKGROUND

Digital watermarking is a process for modifying physical or electronicmedia content to embed a hidden machine-readable code into the media. Indigital watermarking, a media content signal, such as an image or audiosignal, is modified to embed a hidden, digital auxiliary code signalsuch that the auxiliary signal is imperceptible or nearly imperceptibleto the user, yet may be detected through an automated detection process.Most commonly, digital watermarking is applied to media content signalssuch as images, audio signals, and video signals. However, watermarkingmay also be applied to other types of media objects, including documents(e.g., through line, word or character shifting, through backgroundpatterns or tints, etc.), software, multi-dimensional graphics models,and surface textures of objects.

Digital watermarking systems typically have two primary components: anencoder that embeds the watermark in a host media signal, and a decoderthat detects and reads the embedded watermark from a signal suspected ofcontaining a watermark (a suspect signal). The encoder embeds awatermark by subtly altering the host media signal. The readingcomponent analyzes a suspect signal to detect whether a watermark ispresent. In applications where the watermark encodes information, thereader extracts this information from the detected watermark.

Several particular watermarking techniques have been developed. Thereader is presumed to be familiar with the literature in this field.Particular techniques for embedding and detecting imperceptiblewatermarks in media signals are detailed, e.g., in the assignee'sco-pending U.S. patent application Ser. No. 09/503,881 and U.S. Pat. No.6,122,403, which are each hereby incorporated by reference.

Another technology referred to as fingerprinting, in contrast to digitalwatermarking, does not embed auxiliary data in a media signal, butrather, derives a unique content signal identifier from the media signalitself. For some applications where the signal undergoes atransformation in normal use, such as compression, transmission, ordigital to analog to digital conversion, the fingerprint (or host signalcharacteristics used to determined a fingerprint) preferably remainsrelatively unchanged, allowing unique identification of the contentsignal. Fingerprints for a wide selection of media signals may be storedin a database and associated with information or actions to be takenupon detection or calculation of a fingerprint.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a method of identifying songs with audio fingerprints.

FIG. 2 shows an improved method of identifying songs with fingerprints.

FIG. 3 shows a card creation process.

FIG. 4 shows a card usage process.

DETAILED DESCRIPTION

Efficient Embedding of Location Information in Cameras in Combinationwith Cell Phones and GPS Units

Background and Summary of Efficient Embedding of Location Information

As background, we believe that digital asset management (DAM) systemswill mature when the process of entering metadata about content managedin the system is mostly automated—as the searching capabilities arealready pretty good and easy to use. Location information associatedwith media content, e.g., a geo-location of where an image was taken orlocations depicted in the image, is a good example of such metadata.Other metadata may include, e.g., day, time, people depicted in animage, subject matter or descriptive words (“1998 Disneyland Trip”),events, authorship, copyright information, capture device particulars,etc., etc.

We believe that most DAM systems fail, when they do so, due to the timeand effort needed to enter metadata into the system, not because thesystems are too difficult to use.

We can simplify this effort. For example, it would be ideal if locationinformation is automatically embedded in consumer images, automaticallyused to categorize images in DAM systems, and used when searching tofind images from similar locations—thus, increasing the usage of theconsumer based image DAM systems. This will allow consumers to searchtheir images, as well as other's images assuming the consumer is part ofan image sharing or subscription service, based upon location. (Thefile-sharing techniques and systems described in assignee's U.S. patentapplication Ser. No. 09/620,019, filed Jul. 20, 2000 and U.S. PublishedPatent Application No. US 2002-0052885 A1, which are each hereinincorporated by reference, can be readily combined with thesetechniques.) For example, I can search based upon the geo-location ofthe Eiffel tower, and find all of my pictures and my friends picturesthat were taken near the Eiffel tower—or even taken in France—by simplycombining geographic maps of countries (or, e.g., embedded geographicindicators) and location search capabilities. (See, e.g., assignee's USPublished Patent Application Nos. US 2002-0122564 A1; US 2002-0154144A1; and US 2002-0124024 A1, and pending patent applications Ser. Nos.10/423,489 and 10/423,834 for related methods and systems. Each of thesepatent documents is herein incorporated by reference.).

To automatically embed location information from a camera into thepicture, the camera preferably knows where it is located. A simplisticsolution is to put a GPS detector in the camera, but this is often tooexpensive.

A preferred inventive method and system combines the function of cellphones and/or GPS units, which know where they are, with a camera via aphysical or wireless link, so that the camera can inexpensively (e.g.,without itself needing to include a GPS receiver) embed location data.In fact, the system can use a cell phone server so that the camera doesnot even require a GPS receiver.

In one implementation, position data is determined by a cell phone basedon signals received from multiple cell sites (or towers), e.g., based onreceived cell site transmission strength and/or known cell sitepositions. (The location information is relayed (e.g., wirelessly) fromthe cell phone to the camera. Or the camera includes cell phonefunctionality, and communicates with the cell towers itself.).Alternatively, position data is relayed to the cell phone from a cellsite. For example, multiple cell sites can compare respective receptiontimes or received signals for a first transmission from the cell phone.A reception time differential is determined for the first transmissionand is then used to determine a location for the cell phone based onknown locations of the cell sites. Or cell phone transmissions asreceived by multiple cell sites, are compared to determine a position ofthe cell phone, as in triangulation methods. This position data is thencommunicated from a cell site (or tower) to cell phone. The communicatedposition data is optionally encrypted or otherwise scrambled to enhancesecurity.

In another implementation, the camera communicates with a GPS receiverto receive location information.

Detailed Description of Efficient Embedding of Location Information

Embodiment 1—Location Based Upon Cell Phone Tower(s)

Currently, many cell phones are including still and/or video cameras.These cell phones can easily enable video conferencing with videocameras and image sharing with MMS (multimedia messaging services). Aphone can also know where it is located within a cell network by havingits signal strength as received at a plurality of cell towers calculatedand/or compared (and/or a time delay of signal reception, especiallywith CDMA, or Code-Division Multiple Access as provided by QualcommInc., where the system includes a master clock). In alternative positiondetermining technique, one cell tower employs signal directiondetection, such as a circular array of sensors with strength varying inthe circle, to determine the location of a cell phone. This locationinformation can be sent to the cell phone for embedding into the imageor video frame. (See assigned U.S. Published Patent Application No.2003-0040326 A1 for related details on determining a cell phonelocation. This patent document is herein incorporated by reference.). Inother cases, a cell phone determines its location, perhaps by comparingsignals received by multiple cell sites, or by using a timing scheme tomeasure signals. Alternatively, location information can be embedded bya network server into an image or other content, and then sent from acell site to a cell phone during, e.g., multimedia messaging service(MMS) or video conferencing. In fact, an image can be uploaded to theserver from a cell phone, embedded with the cell phone's locationinformation, and then downloaded back to the phone, even if not used forMMS or video conferencing. This means that the cell phone/camera do notneed to include a watermark embedder.

The embedding process can include watermarking or out of band methods,such as header tagging. The location of the cell phone and/or cameracombination during the capture of the image can be redundantly embeddedthroughout the image, or more complex calculations can be done to embedthe location of the image (or objects or areas depicted in an image)based upon the focal point of the camera, which includes a calculationincluding the cell phone/camera location, angle, focus length, and zoom,for example. Or, perhaps, only the location of the focal point of thecenter of the picture may be embedded, or the image can be broken intosmaller section with each section containing local location information.

Embedding the complete location information in each section can embedthe local (or area specific) location information. Alternatively, thelocal location information can be embedded by embedding the centrallocation and differential change of the focal point (or cell phonecamera combination location) into a group of sections, and each sectionin that group contains an embedded differential code. Thus, the locationof that section can be calculated by adding the differential code timesthe differential change to the central location. This group may be partof a complete picture. The group may include a predetermineddifferential size, such that only the central location needs to beembedded in the group, along with each section having a differentialcode.

The cell phone and camera can be one device (as many cell phones nowhave cameras onboard) or separate, but physically (or wirelessly)connected devices. In either case, the connection, whether internal orexternal, between the cell phone and camera could be proprietary orbased upon a standard such as USB or BlueTooth.

Location information can be shared between a cell phone and camera basedupon using multimedia platforms, such as Qualcomm's BREW, or requiring acell phone modification.

If a network server does the embedding, the cell phone preferablyincludes a control module (e.g., software or firmware) to helpfacilitate the cell phone to upload and then receive each embeddedimage. As such, this system requires minimal cost to implement.

Embodiment 2—Linking Camera and GPS

In a consumer device world which is moving from physically connected towireless networking, such as via BlueTooth or IEEE 802.11b (a.k.a.WiFi), a camera and location device, such as cell phone or GPS unit, donot have to be physically connected. The camera and cell phone can actas described in embodiment 1 but via a wireless connection.Alternatively, a GPS unit with wireless capabilities can be used withthe camera to provide location information.

As such, the consumer whom has a camera, and cell phone or GPS system,does not need to buy a combination. Thus, the consumer will not end upwith a multitude of devices, or several GPS locators embedded withineach device.

In some of the above embodiments, a server can determine a location andcreate bits to be embedded for a pre-determined image size. Forpseudo-random noise (PN) watermarking techniques, this may includemultiplying a watermark payload, including error correction, by the PNsequence, and including an orientation and synchronization signal, ifapplicable. The server can then download this watermark signal to a cellphone, which can, in turn, add it to the image or perform human visualmodeling to reduce visibility of the watermark, preferably using anefficient human visual model. Of course, an image does not need to beuploaded, thus saving bandwidth and consumer expense, while the cellphone requires less CPU power to embed the watermark.

In addition, in both embodiments, a picture location can have time andbiometrics added to determine where, when and who (e.g., who's depictedin the picture and who took the picture). The camera or server canidentify people in the picture and who took the picture, via retinal oriris scanning in the eyepiece, fingerprint recognition in the shutterbutton, or face recognition using an image of the camera user. For adescription of a capture device that captures biometrics of the user andembeds the biometric data in images, audio or video captured by thedevice, see e.g., U.S. patent application Ser. No. 09/585,678, which isherein incorporated by reference.

Watermarked Participant IDs used for Proper Billing

Background and Summary for Participant IDs

In distribution of content, especially digital content, it is sometimesdifficult to determine which parties require royalty payments. Audioroyalties can be based upon the usage, such as whether the content isplayed on the radio, or as background music during a radio or TV show,etc. In addition, royalties include parties involved in both theperforming and recording process (and can be extended to distributionand retail partners outside the area of copyright, as described below).There are often at least two copyrights associated with a particularpiece of music, e.g., performance and recording. Video can be just asdifficult, especially for ads where actors have to be properlycompensated.

If a watermark in the content includes IDs for all parties that shouldreceive royalties, the system of determining who to pay duringdistribution, especially digital file sharing, is made easier. Biometricanalysis, such as facial or voice recognition may be used to identifypeople in images, video and audio to assist in determining appropriateparties to which royalties are due.

A similar problem occurs for content bought from a retailer (ordistribution service provider acting as a retailer) that is allowed tobe super or re-distributed by the consumer, such as content that isallowed to be shared on a file sharing system with a billingmethodology. The retailer should be compensated, as well as copyrightholders, whether or not, but optimally, if the file sharing systemreceives revenue from the content. For example, this can be audio sharedon KazaA or VOD video shared with a SonicBlue Replay 4000™ PVR (wherethe retailer is the VOD service provider).

If the content includes a retailer (or distributor/VOD) ID, then thefile sharing system can identify the original retailer and compensatethem. The compensation can be per download, per rendering (e.g., perlistening or viewing), or a percentage of royalty related to that song.More specifically, the retailer could be compensated with a percentageof revenue of the P2P provider related to the percentage of time thatthe retailer's song was downloaded. Similar royalties can relate tocopyright owners, such as artists and record labels.

Detailed Description for Participant IDs

The digital watermarking system is a multi-step process (with each stepsometimes having multiple sub-steps):

-   1. Each participant registers for a participant ID, and the ID is    linked to the participant in a database (e.g., via a web based    interface to a server or group of servers on the Internet or other    computer network).-   2. The content is embedded with the appropriate participant IDs.-   3. The detection system reads the participant IDs, links to the    database to interpret them, and enables the correct response or    billing action.

Each participant preferably carries the first step out only once. Thedatabase can be centrally located with all participant IDs, centrallydistributed, either mirrored or intelligently distributed, or local andupdated (see, e.g., assignee's U.S. patent application Ser. No.10/060,049 (published as US 2002-0162118 A1) and related PCT ApplicationNo. PCT/IUS02/02572 (published as WO 02/062009), each of which is hereinincorporated by reference).

The second step needs to be only done once for each piece of content.The watermark can be chosen from a multitude of watermarking methodsdepending upon the content format, desired robustness, payload size,embedder and detector computational complexity, and perceptibility.

The third step is preferably performed each time the content goesthrough a transaction. For example, the third step may not have to occurduring a download, if the song has limited functionality for evaluation,but should occur when the song is purchased or enabled for longerevaluation.

The third step may involve several sub-steps. For example, theparticipant ID is extracted from the content. Then, the database is usedto link the participant ID to the participant. Next, an appropriateaction is enabled.

The appropriate action may include a multitude of actions and relatedsub-steps. The appropriate action may be crediting an artist or retailerinside a P2P system or related billing system. The participant is paid,either each time content is transacted (e.g., a micro-payment) or as alarger transaction based upon the time from the last payment (e.g., payevery month) or the amount credited (e.g., pay when credit reaches$100).

The appropriate action may include logging the participant ID so that itcan be submitted to a collection agency, such as BMI, ASCAP, and SESAC,and used by these agencies to determine how to distribute royalties. Arecord label, for promotion and demography research, may use the system.Or the system may be used to inter-participant trading value, such aswithin bartering systems.

Improved Broadcast Monitoring based upon Reduced Fingerprint DatabaseSize

Background and Summary for Improved Broadcast Monitoring

Content fingerprints, which are methods to identify content based uponthe content itself, typically use a database to perform some processingto match incoming fingerprint to fingerprints stored in the database.The larger the database, the more processing required by the database,and the less accurate the system.

Content fingerprinting methods can be used to monitor broadcasts, suchas radio and TV. A dilemma is as follows. To monitor more stations, thesystem may need a large database of songs, which causes the system to beslower and less accurate. However, when monitoring fewer stations, thesystem is less economically attractive as a business because, once thefingerprinting system is developed and implemented, it is cheaper tomonitor more stations as the only added cost is another radio receiver.

The novel solution is, for each station that is being monitored, thesystem only searches a database for content played on that station, suchas songs played on that radio station during a predetermined period(e.g., a week, etc). The radio and TV stations have play lists that canbe provided to the monitoring services.

Detailed Description for Improved Broadcast Monitoring

This system is applicable to radio and TV. The detailed example is shownfor audio, since these systems have currently shown to be morepractical. However, our inventive techniques can be applied to othertypes of content, such as video as well. (When considering videobroadcast monitoring systems, or if audio fingerprinting systems areused to monitor TV stations, a TV play list will replaces a radiostation song play list and the system works as described for radiobroadcast monitoring.)

FIG. 1 shows a conventional method of identifying songs withfingerprints. Step 300 calculates the fingerprint for the song orsection of the song.

Step 310 compares the calculated fingerprint to the database 350 andfinds the closest match. This step may involve some type of distancecalculation between the calculated fingerprints (sometimes includingsub-fingerprints) and the database entries, which is computationallyintensive and increases the likelihood of error as the database becomeslarger. Even in fingerprint methods (potentially created in the future)that do not involve distance measurements, a larger database increasesthe likelihood of error. For example, if the fingerprint ID is erroneousdue to background noise or a voice over, it is more likely to match adatabase ID in a larger database.

The third step involves identifying the song once the song's fingerprintis matched in the database.

FIG. 2 shows an inventive method of identifying songs with fingerprints,for systems such as radio monitoring, where a limited play list can beused to limit the database search space. The first step 300 is identicalto the currently used method, where the song's fingerprint iscalculated.

Step 330 includes comparing the song's fingerprint to a database (ordata repository), which is limited to a play list of the radio station(or equivalent audio service, such as an Internet streaming system). Thedatabase used in step 330 is, e.g., preferably a selection of the songsfrom the radio station play list 340, which is a subset of the completedatabase 350.

The database used in step 330 may be local to the fingerprinting system,such as located on the same PC as calculating the fingerprint. The localdatabase should be updated as the radio station play list changes.Alternatively, the database may be accessible via a network, such as theinternet in a central database, or mirror or intelligently distributeddatabase method. The intelligently distributed database method is aninitial step of limiting number of database entries, where, for example,US songs not played in Europe do not exist in the versions of thedatabase in Europe.

Wherever the database is located, the database can have only entries forsongs included in the play list, or songs in the play list arehighlighted as active. The latter means only the active entries arechanged when the play list is updated, whereas the former means thedatabase entries for newly added songs have to be added and databaseentries for songs in the database not included in the new play list haveto be deleted.

The play list does not need to include times that the song is played,just a total list of every song played for a period of time, such asthat week or month. The play list may have a likelihood or number oftimes each song is played to help the fingerprint system choose betweentwo close matches.

As such, the song identification step 320 is now more accurate, and mostlikely more efficient, since the database used in step 330 is smallerthan the database used in step 310.

Fair-Use via Watermark Continuity Measurements

Background and Summary for Fair-Use via Watermark ContinuityMeasurements

United States Copyright law, as well as other copyright laws, such as inEurope and Asia, provide fair use safe harbors. Fair use usually enablescertain people to use certain amounts of content without requiring topay the copyright owner. For example, in the US, a teacher can use ashort segment of a movie if certain fair use criteria are met.

In our digital world it is easy to obtain content and hard to determinefair use, so fair use is sometimes ignored. In addition, there arecontent security systems that are being built that violate fair use. Assuch, a technical method that enables calculation or determination offair use is desirable.

One prior technique to provide fair-use is disclosed in U.S. Pat. No.5,765,152 (Erickson), which is herein incorporated by reference. In oneembodiment of the Erickson patent, media content is bundled in acontainer including minimum permissions (e.g., to allow fair use of thecontent). The media content can be handled only according to the minimumpermissions.

My proposal, in contrast, uses a digital watermark embedded in contentthat is used to measure contiguous time the content is rendered orviewed. A playing or rendering system uses the measure to determine fairuse, and can limit contiguous access to protected content within thesystem once the fair use limits have been exhausted.

Detailed Description for Fair-Use via Watermark Continuity Measurements

Before content distribution, a watermark is embedded in the content witha payload including a static copy protection bit or bits, a staticcontent ID, and dynamic counter or time reference indicator that can beused to determine the length of content. A rendering system used inlocations requiring fair use, such as schools or research institutions,include a watermark detector to measures contiguous time that protectedcontent is played. If the time is too long, or the content is used in amanner failing outside of fair use's boundaries, the rendering systemstops working because fair use has been violated. The content ID is usedto determine whether the play is contiguous and is not from differentsegments of protected content. (The protection bits can also be used tosignal that the content is protected content. In some implementations,the copy protection bit(s) and the time interval indicator comprise thesame payload bits. Of course, in other implementations, the copyprotection bit(s) and the time interval indicator are separate payloadfields.).

Micro-Payment via Watermark Security Cards

Background and Summary for Micro-Payments

In a secure payment system “Pay by Touch” by Indivos Inc. of Oakland,Calif., payments are made based upon a person's fingerprint (from aperson's finger, not to be confused with a song's fingerprint asdescribed above). The system involves a quick sign-up process, whichincludes the calculation and storage of the person's fingerprint. Acheckout procedure involves a customer placing her finger on a touch padat the checkout and entering a phone number or ZIP code. The readercompares the fingerprint scan to the images on file and calls up theappropriate account number. After the transaction authorization, thecustomer must also sign the sales slip.

This type of security can alternatively, and preferably, be provided viaa secure driver's licenses or security card based upon digitalwatermarks. The watermarks authenticate the card originality as well asthe picture as matching the card data. The debt could be accumulated andsubmitted every week or when it went above $100. By submitting largeramounts, the retailer can saves money in terms of fixed fees for creditcard transactions, as well as being able to negotiate a lower rate dueto an increased number of larger transactions. However, the retailer isassuming debt, and this amount can be based upon personal knowledge ofthe consumer (if legal) and/or past shopping history.

Detailed Description for Micro-Payment Improvements

Two processes are discussed: one for the creation of a customer card andone for the usage of the card at a point of sale.

As shown in FIG. 3, the card creation process receives, in step 400,personal information from a customer. For example, a retailer can obtainthe customer's name, address, phone number, and credit card or bankinformation. The personal information can be confirmed by a credit cardcompany, as currently done with many currently used systems, such aswhen ordering on the Internet.

Step 410 includes obtaining the person's picture. The picture can bedirectly captured (e.g., via a digital camera) or obtained from aphotographic repository.

Step 420 includes creating and embedding a digital watermark. A robustwatermark to be embedded in the customer picture is preferably basedupon the personal information; that is, a digital watermark payload iscreated and embedded in the picture that somehow links the picture tothe information on the card or to information provided by the customer.The payload may include, for example, a 20-bit MD5 hash of the name,address and phone number. The likelihood that these 20 bits are notunique for people who look similar is low enough to provide adequatesecurity. In addition, within the picture or background of the card, afragile watermark can be added that can identify whether the card isoriginal or a copy.

The card is printed in step 430. Most any printer can be used as mostprinters usually have minimal effects on watermarks, but the most securesystem will include a list of recommended card printers. This list willinclude printers that most accurately represent the watermarks on thecard.

As shown in FIG. 4, the card usage process is employed, e.g., in step440, where, at a checkout, a card reader authenticates the card. Thecard reader may be part of a kiosk with a window to place the card, maybe a stand alone or tethered hand-held reader, or a device that the cardis slid into and pulled out or automatically returned (i.e. a automaticscanner). Typically, the reader will include an optical sensor tocapture optical scan data representing the card or a portion of the card(e.g., the picture). A digital watermark decoder analyses the scan datato decode the watermark and obtain the watermark payload.

If the reader is part of a kiosk, the kiosk system may also be used tohelp link the user to previous buying habits and pricing. For example,the user may be able to look up how much they paid for milk a month ago,as well as how much milk they have bought in the last month. Inaddition, they may be able to look up warranty information for productsbought at a store via the kiosk. Similarly, if the store allows onlinebrowsing, the kiosk can link to the online shopping. For example, thekiosk can link to a wish created while shopping online. This linking isfacilitated, at least in part, by the decoded digital watermark. Forexample, the payload includes an identifier—perhaps a hash of acustomer's personal information—which is used to link to informationregarding the customer. (See, e.g., assignee's U.S. patent applicationSer. No. 09/571,422, filed May 15, 2000, which is herein incorporated byreference, for additional watermark-based linking techniques.)

Since the ID created from the hash may not be entirely unique and thekiosk system may sometimes need a unique ID for each user, a PIN and/orperson's name can be used to guarantee that the ID is unique.Alternatively, a seperate unique ID can be added to the image orbackground of the card based upon a central registration process duringthe creation of the card to obtain the unique ID.

In step 450, a store employee can optionally look at the card and holderto make sure the picture matches the person. Optionally, in addition toor instead of comparing the picture to the person, a PIN (personalidentification number) or biometric sample can be entered. The idealsecurity system involves something you have (i.e. ID card), somethingyou are (e.g., biometric sample like a fingerprint, voice print, retinalscan, etc.), and something you know (e.g., PIN or sequence). The card issomething you have. Comparing the picture (or biometric) to the carduser is something you are, and requiring a PIN requires something youknow. Thus, if both optional steps are included, where each stepinvolves minimal cost, the system is more secure than just requiring acard or fingerprint.

In step 460, the retailer adds the price of the purchase to an accountor data record associated with the customer, thus accumulating thecustomer's debt. The retailer can accumulate debt for a pre-determinedamount of time and/or up to a pre-determined amount. The pre-determinedamount can be dependent, e.g., upon a customer's past history ofshopping at the store, her credit history, or her personal relationshipwith the retailer (subject, of course, to any applicable laws).Pre-determined amounts can change over time, dependent upon thecustomer's usage patterns and how quickly she pays the store or heraccount.

As such, the retailer is accumulating risk, balanced by saving expensesby dealing with many small transactions with credit card companies orbanks. For example, a user may disappear and cancel a credit card beforethe retailer bills the credit card—which leaves the retailer out ofmoney.

The pre-determined amounts can work similar to how credit cards increasea credit limit over time. For example, for the first 6 months, thesystem may submit every week or when an account reaches $50, which everis first. Then, assuming the person continues spending money and payingtheir debt, the values can be increased to 10 days and $75, and so on.

At the correct time in step 470, based upon the previously describedrules, the debt is submitted to the credit card company or otherfinancial institution.

Alternatively, rather than using a credit card company, the debt couldbe billed to the user directly, or highlighted for the next time theuser arrives for payment (especially if the retailer is part of a user'sclub or co-op where the user visits regularly).

This system saves the most money for retailers that have repeatcustomers who spend minimal amounts each visit, such as for aneighborhood market or convenience store, or hardware store.

This system can also be used with an Internet online retailer, since aPC and camera can securely read a watermarked card. In fact, thepre-determined amounts can be updated by ratings of the Internet sitefrom other user if the system involves trading, such as for Web siteslike eBay.com.

This system could also use a driver's license, when the driver's licensehas digital watermarks authenticating the card, instead of a proprietarystore card. Similarly, instead of the card, the fingerprint can be usedwith the described debt accumulation system. The fingerprint canpotentially along with the PIN, signature and/or even a security cardwith a watermark or magnetic strip (for something you have), could beused with the method for accumulating micro-payments. In this system,the fingerprint and signature (if included) are something you are, thePIN (if included) is something you know, and security card (if included)is something you have. Thus, if the optional parts are required, thesystem has maximum security.

Digimarc MediaBridge enabled Physical Internet Access Logon Cards

Background and Summary for Access Logon Cards

Many Internet cafes and hotels with wireless networks and/or centralInternet-ready PCs use passwords to stop unauthorized people from usingan Internet link, as well as to track usage. However, it is easy forusers to share the passwords, as well as it is expensive to maintainbecause the system requires a hotel to change, remember, and provide thecurrent password.

A logon card including a digital watermark embedded therein that can beread by a PC optical camera, can enable Internet access in computerswith cameras. One type of watermarking scheme is provided by Digimarc's(Tualatin, Oreg., USA) MediaBridge™ technology. Our logon cards are moresecure than passwords since when people share them, a copy is not made.These logon cards are more efficient than passwords because they don'tneed to be changed since they cannot be duplicated. The logon card canbe combined with existing cards, such as with modern plastic hotel roomkeys (that use a magnetic strip to open the door) or with personalmembership cards. This combination reduces cost because several cardsdon't have to be created and monitored for loss.

Detailed Description for Access Logon Cards

Detailed examples involving a hotel and Internet cafe are one efficientway to explain this system.

For a hotel with a magnetically coded room keys, if every room key ispre-watermarked with a unique ID, when the key is given to the user andcoded for their room, the watermark ID can be linked to the room. Then,if the consumer wants to use a computer in the business center of thehotel, for example, rather than requiring a password, the user showshis/her watermarked room key to the web camera on the PC in the businesscenter, and the computer is unlocked so the consumer can use it. Sincethe room key is linked to the room, the computer usage can beautomatically billed to the user's room.

Similarly, our logon card system can be employed for consumers to use awireless network—which works as follows. A user shows there watermarkedroom key to a wireless access terminal. The access terminal captures animage of the room key, decodes the watermark from captured image, checkto see if the ID is valid (e.g., ensuring the that user is a currentquest at the hotel), and then enables wireless access, if the ID ivvalid. (The hotel or affiliated provider can maintain a database ofunique ID. IDs can be flagged valid or invalid, or even removed from thedatabase if they have expired.) Thus, consumers with wireless networkcards do not need encryption codes that can easily be shared and hard tofind at the front desk to use the hotel's wireless network internetconnection, they only need a PC with a camera. In turn, the hoteldoesn't need to maintain the encryption system.

Once again, the system can automatically bill usage to the user's roomsince the watermark is linked to the hotel room, even though theconsumer can work as they wonder throughout the hotel (e.g., not lockedin their room or a hotel business center).

This system can also work for in-room wired networks. Although, the roomnumber can be known in this case due to the physical wire connected tothe room, if the hotel is controlling the PCs in its business center andwireless network with the room key, it may be advantageous to have allthe systems work identically. In addition, with our logon card system,the rooms can be wired with standard ethernet capabilities, where thephysical wires don't have to be linked to a room (and the PC doesn'thave to be identified so several people don't share a connection in oneroom).

In an Internet cafe example, a logon card given to the consumer whenthey are ready to use a PC controls login. The logon card is shown to acamera on the PC to log them on, as described for the PCs in the hotel'sbusiness center. The usage can be timed on the PC that the consumer isusing (as identified by a watermark ID on the logon card), or by amaster PC, which determines when the logon card was given to theconsumer and when returned (by showing it to the master PC's camera whengiven and received). With either billing method, even if the consumershares the logon card, they are still billed since it can identify them.

In addition, if an Internet cafe wants to have a membership card, wherea consumer gets 5 free hours with every 20 hours used, for example, themembership card can be watermarked with a unique ID that is used to logonto the computer and link to the member. The card can also havesecurity measures, if desired, such as fragile watermarks to locatecopies and watermarks that link the picture to the personal data (viaembedding a 20 bit hash of the user's name, for example). The fragilewatermark degrades when a reproduction of the originally watermarkeditem is made, enabling a watermark reader to differentiate reproductionsfrom an original, watermarked object.

The system level details include that after the card is shown to the PCwith the camera, the watermark ID is sent to a database that links theID to a room number and identifies that the ID is active. If the ID isactive, the consumer is allowed to logon, and the appropriate billingaction is started. If the ID is not active, the logon is not allowed.They system should be secured by authenticating the card readingsoftware to the database, and encrypting the watermark ID when sentwithin the PC or network. Session keys should be changed to be resistantto replay attack, as well known in the art of cryptography.

In the Internet cafe system, which uses a master PC to monitor theamount of time the card is checked-out, the PC that the consumer usesonly needs to verify that the watermark ID is active or authentic toenable logon.

See, e.g., assignee's U.S. patent application Ser. Nos. 10/382,359 and09/571,422, which are each herein incorporated by reference, for relatedtechniques and/or environments.

Surveillance Video Authentication

Background and Summary for Surveillance Video Authentication

Authentication of surveillance video, such as that captured by closedcircuit TV (CCTV), is important to verify in court that the surveillancevideo is authentic. The Digital Signature Standard (DSS), or any digitalsignature (by definition, including the private key encryption of arobust hash), can be used to authenticate the accuracy of every bit ineach frame of surveillance video. The unique combination of private keyusage, frame splitting and date-time addition can improve theauthentication to guarantee that no frames were removed, that the frameswere recorded at the appropriate date and time, and that the appropriaterecorder performed the recording. Location information can also beembedded in the video in the form of a digital watermark to improveauthentication.

Detailed Description for Surveillance Video Authentication

The DSS, as described in Spec “Federal Information Processing Standard(FIPS186) Digital Signature Standard (DSS)” available at<http://www.itl.nist.gov/fipspubs/fip186.htm> and uses the secure hashalgorithm (SHA-1) as described at<http://www.itl.nist.gov/fipspubs/fip180-1.htm>, or any other digitalsignature based upon a robust hash and public key cryptography, is usedto demonstrate that no bits in each frame have been modified. Thesignature for each frame is saved as a signature frame in a separatefile or part of the video header, with the whole signature saved in thevideo header or each signature frame saved in each corresponding videoframe's header. The system also demonstrates that no frames have beenadded because the correct signature cannot be calculated.

As an alternative and inventive option, the system can use half of oneframe and the other half of the next frame in the signature for eachframe. This system demonstrates that every bit of each frame isauthentic and that no frames have been added (as before, as well as thefact that no frames have been removed). The additional authenticationthat no frames have been removed is based on the fact that each framesignature includes bits from itself and another frame; thus, if anyframe is removed the previous frame's signature will not match. Thissystem can be designed in many ways, such as using thirds of a frame,the previous frame and the next frame, as long as every bit of eachframe is included somewhere in the signature and one frame is notcompletely included in one signature frame. The signature frames can besaved in a separate file, as part of the video header, or as part ofeach frame header.

Alternatively, the complete group of signature frames can be encryptedwith the private key to make sure no frames are removed. Similarly, thesystem, for a fixed video, could consider all the frames as one largemessage, and perform a DSS on the complete message, as long as themessage length does not exceed the limit set by the digital signature.

Additionally, the date and time (and/or geo-location associated with thevideo or area depicted in the video) can be included as part of theframe signature to verify the date and time (and/or location) of thevideo. The date time (and/or location) can be added to the robust hashcalculation (by converting the date time stamp to bits and included asany additional frame bits) or appending the date time stamp to therobust hash payload and then encrypted with the private key. The datetime stamp can have frame or second accuracy (or maybe even minuteaccuracy).

Furthermore, each video recorder can share a system-level private key orhave its owner recorder private key. If separate recorder private keysare used, the system can track the machine from which the video wasrecorded based upon the one-to-one relationship of the decryption publickey and encryption private key. More specifically, as well known, if thewrong public key is used, the authentication process will fail; thus,the public key identifies the private key used, which, in turn,identifies the machine.

As discussed in the DSS or well known in the state of art forcryptography and digital signatures, the authentication process includesusing the public key to decrypt the digital signature and compare it tothe robust hash calculation of the video frame data (and possibly dateand time, if included as part of the hash and not a separate part of thepayload). If they match, the video frame data is authentic. If the datetime stamp is included as an appended part of the robust hash payload,the date time stamp can be read from the digital signature to verify thedate and time.

As such, if every additional option is used, every bit of the video isshown to be authentic, meaning that no bits have not been modified,removed or added. In addition, the date and time of the video is known.Finally, the recorder can be identified based upon the public decryptionkey.

Alternatively, the RSA algorithm, rather than DSA algorithm can bechosen, as described at<http://www.rsasecurity.com/rsalabs/faq/3-4-1.html>, to make theauthentication (referred to as verification in the referred web page)process faster than the signing process, but this is probably notadvantageous for this system since the signing must be done in real timeon the recorder and the authentication can be done offline.

Furthermore, small segments (like 128×128 pixels) of each frame can beused, with some segment overlapping between frames. This has theadvantage of demonstrating the location of any manipulation, as well aslinking frames together, so frames cannot be removed. The digitalsignature for each segment within each frame and for each segmentstarted within the frame (and including some of the next or otherframe), can be stored in the header of that frame or as a group offrames, and one needs to include a date time stamp. In other words, theframe digital signature consists of many smaller signatures, andincludes a date time stamp.

The digital signatures could be included in reversible watermarks withinthe frame content for each frame segment, rather in header data. Areversible watermark is generally a watermark, which can be removed fromcontent without degrading (or without significantly degrading) thecontent. In some cases, removing a watermark implies restoring contentto its unmarked state. Suitable reversible watermarks are described,e.g., in assignee's pending U.S. patent application Ser. No. 10/319,404,filed Dec. 12, 2002 and Ser. No. 10/435,517, filed May 8, 2003, whichare each hereby incorporated by reference.

Concluding Remarks

Having described and illustrated the principles of the technology withreference to specific implementations, it will be recognized that thetechnology can be implemented in many other, different, forms.

To provide a comprehensive disclosure without unduly lengthening thespecification, applicants incorporate by reference the patents andpatent applications referenced above.

The methods, processes, and systems described above may be implementedin hardware, software or a combination of hardware and software. Forexample, the auxiliary data encoding processes may be implemented in aprogrammable computer or a special purpose digital circuit. Similarly,auxiliary data decoding and fingerprint calculation may be implementedin software, firmware, hardware, or combinations of software, firmwareand hardware. The methods and processes described above may beimplemented in programs executed from a system's memory (a computerreadable medium, such as an electronic, optical or magnetic storagedevice).

The particular combinations of elements and features in theabove-detailed embodiments are exemplary only; the interchanging andsubstitution of these teachings with other teachings in this and theincorporated-by-reference patents/applications are also contemplated.

1. A method of steganographically embedding geo-location information inan image captured by a camera associated with a cell phone, said methodcomprises: determining geo-location information based on attributesassociated with a cell phone network; and steganographically embeddingthe geo-location information in the image.
 2. The method of claim 1,wherein the cell phone network comprises a signal tower, and theattributes are associated with the tower.
 3. The method of claim 2,wherein the cell phone communicates a signal, and wherein the attributescomprise strength of the signal as received by the tower and a directionassociated with the signal.
 4. The method of claim 1, wherein the cellphone network comprises a plurality of towers, and wherein the cellphone communicates a signal, the attributes comprising an evaluation ofthe signal as received by the plurality of towers.
 5. The method ofclaim 4, wherein the evaluation considers relative reception timing ofthe signal as received by each of the plurality of towers.
 6. The methodof claim 4, wherein the evaluation involves triangulation.
 7. The methodof claim 1, wherein the cell phone comprises a steganographic embedder,and uses the embedder to steganographically embed the geo-locationinformation in the image.
 8. The method of claim 1, wherein the cellphone network comprises a steganographic embedder located remotely fromthe cell phone, and wherein the embedder steganographically embeds thegeo-location information in the image.
 9. The method of claim 8, furthercomprising communicating the embedded image to the cell phone.
 10. Themethod of claim 1, wherein the steganographic embedding comprisesdigital watermarking.
 11. A method of steganographically embeddinggeo-location information in an image captured by a camera which isintegrated with a cell phone, wherein the cell phone comprises awireless interface, said method comprises: communicating with a globalpositioning system (GPS) receiver, which is remotely located from thecell phone via the wireless interface; receiving geo-locationinformation from the GPS receiver; and steganographically embedding thegeo-location information in the image.
 12. The method of claim 11,wherein the geo-location information and image are communicated to acell phone network server which includes a steganographic embedder, andwherein the cell phone network server performs said step ofsteganographically embedding the geo-location information in the image.13. A method of providing internet access for a computer usercomprising: issuing the user a digital watermarked object, wherein thedigital watermarked object comprises a digital watermark embeddedtherein, the digital watermark comprising an identifier; associating theidentifier with the user via a data repository; receiving optical scandata corresponding to a portion of the object, the portion comprisingthe digital watermark; decoding the digital watermark from the scan datato obtain the identifier; verifying that the identifier is valid; andenabling internet access for the user when the identifier is valid. 14.The method of claim 13, wherein the object comprises at least one of ahotel room key and an object provided by a hotel.
 15. The method ofclaim 14, further comprising associating a bill for internet access withthe user via the identifier.
 16. A method of accumulating financialcharges attributable to a customer so as to minimize transaction fees,the customer has possession of a digitally watermarked object, thedigitally watermarked object comprising a digital watermark including anidentifier, said method comprising: receiving scan data associated withthe digitally watermarked object; analyzing the scan data to obtain theidentifier from the object; accessing a data record that is associatedwith the identifier; updating the data record to reflect a monetaryamount owed for a transaction; accumulating a plurality of such monetaryamounts in the data record; and forwarding the accumulated amounts forpayment at least when one of the following occur: a predetermined amountfor the accumulated amount is reached or after a predetermined amount oftime.
 17. The method of claim 16, wherein the aggregated monetaryamounts are forwarded to the customer for payment.
 18. The method ofclaim 16, wherein the aggregated monetary amounts are forwarded to acredit agency for payment.
 19. The method of claim 18, wherein thecredit agency comprises at least one of a bank or credit card company.20. The method of claim 16, wherein the identifier comprises informationpersonal to the customer.
 21. The method of claim 20, wherein theidentifier comprises a hash of the personal information.
 22. The methodof claim 16, wherein the identifier is combined with informationprovided by the customer to access the data record that is associatedwith the identifier.
 23. The method of claim 16, wherein the digitalwatermark further comprises a biometric, and said method furthercomprises comparing a biometric sample of the customer to the biometriccarried by the digital watermark.
 24. A method to regulate protectedcontent while allowing fair use of the content, wherein the contentincludes a digital watermarking embedded therein, the digitalwatermarking including at least a copy protection indicator and a timeinterval indicator, said method comprising: recognizing the content asprotected content by reference to the copy protection indicator; andupon recognition of the content as protected content, measuring theamount of content rendered by reference to the time interval indicator,and disabling rendering after a predetermined amount of content has beenrendered, the predetermined amount corresponding to predetermined fairuse of the content.
 25. A method for providing royalty payments forcontent distributed via a network, said method comprising: receivingregistration information from a participant who requires royaltypayments for content to be distributed; assigning a unique identifier tothe participant; steganographically embedding the content with theidentifier; and associating a royalty payment action with the identifierin a data repository.
 26. The method of claim 25, wherein the royaltypayment action is initiated when a rendering device decodes thesteganographic embedding and obtains the identifier during atransaction, the identifier being provided to the data repository and inresponse, said method comprises performing the royalty payment action,wherein the transaction exceeds evaluation of the content.
 27. Themethod of claim 26, wherein the network comprises a peer-to-peerfile-sharing network.
 28. The method of claim 27, wherein the royaltypayment action comprises determining a percentage of revenue thatcorresponds to an amount of times the content undergoes a transaction.